OS Kernel

The basic virtual memory-mapped communication (VMMC) model provides protected, direct communication between the sender's and receiver's virtual address spaces, but it does not support high-level connection-oriented communication APIs well. This paper presents VMMC-2, an extension to the basic VMMC. We describe the design, implementation , and evaluate the performance of three mechanisms in VMMC-2: (1) a user-managed TLB mechanism for address translation which enables user libraries to dynamically manage the amount of pinned space and requires only driver support from many operating systems; (2) a transfer redirection mechanism which avoids copying on the re-ceiver's side; (3) a reliable communication protocol at the data link layer which avoids copying on the sender's side. To validate our extensions we implemented stream sockets on top of the VMMC-2 running on a Myrinet network of Pentium PCs. This zero-copy sockets implementation provides a maximum bandwidth of over 84 Mbytes/s and a one-way latency of 20 s. 1 Introduction With the arrival of fast networks such as Myrinet and ATM, the overhead of the network communication software has become a major performance bottleneck, hurting user-to-user latency and bandwidth. For example, the TCP/IP protocol overhead increases the latency by several hundreds microsec-onds 24] even for small messages. A copy in the network protocol can reduce the communication bandwidth to the memory copy bandwidth which is usually substantially less than the peak hardware DMA bandwidth. For example, while the Myrinet hardware is capable of delivering band-width of 1.28 Gbits/s 7], the end-user applications see little performance improvement compared to the 100 Mbits/s Ethernet 25]. The desire to build high-performance servers from a network of commodity computers has pushed researchers to explore ways to reduce the communication bottleneck for system area networks. During the last few years, researchers have proposed several approaches to move network protocols partially or entirely to the user level with protection in order to reduce the protocol overhead 3, 15, 18, 19, 36]. The common framework is to implement a user-level data transport protocol that integrates well with the network interface. Existing high-level protocols for shared virtual memory 21] and message-passing 1, 5, 14, 32] can then be implemented